Performance Effects of a Cache Miss Handling Architecture in a Multi-core Processor

Multi-core processors, also called Chip multiprocessors (CMPs), have recently been proposed to counter several of the problems associated with modern superscalar microprocessors: limited instruction level parallelism (ILP), high power consumption and large design complexity. However, the performance gap between a processor core and main memory is large and growing. Consequently, multi-core architectures must invest in techniques to hide the large memory latency. One way of doing this is to use non-blocking or lockup-free caches. The key idea is that a cache can continue to service requests while one or more misses are being processed at a lower memory hierarchy level. This technique was first proposed by Kroft [16]. The main contribution of this paper is the observation that a non-blocking cache must fulfill two functions. Firstly, it should provide sufficient miss parallelism to speed up the applications. Secondly, the number of parallel misses should not be so large that it creates congestion in the on-chip interconnect or off-chip memory bus. While the first function is well known, the other is a result of the possibility for contention when multiple processors are placed on a single chip. A compromise miss handling architecture (MHA) evaluated in this work which handles 16 parallel misses in the L1 cache, has an average speed-up of 47 % compared to a blocking cache and has a hardware cost of 9 % of the L1 cache area.